Trim server-loop overhead and add a socket NIF protocol - #139
Merged
Conversation
Keep the in-flight request queue in the server state as a map instead of a shared ETS table; each request paid two ETS copies plus table lock traffic for data only its own process touches. Reply messages now carry just the request id instead of the whole cast record, and telemetry can be disabled with the shackle 'telemetry' app env, skipping the handler-table lookup on every event. Benchmarked at 64 callers / pool of 16 against the arithmetic test server: 156-158k -> 167-168k requests/s (+7%).
An alternative to shackle_tcp built on the socket module. Accepted
sockets run with {otp, select_read}, so the server gets one $socket
message plus one recv per wake instead of the inet driver's active
mode delivery, and send is a single NIF call instead of a port
command with its monitor and inet_reply round trip.
Benchmarked at 64 callers / pool of 16 against the arithmetic test
server: 168-169k requests/s with shackle_tcp, 172-174k with
shackle_socket (+2.5%). Requires OTP 27.3 for select_read; the
default protocol is unchanged.
With the JIT and module-wide type propagation these no longer have a measurable effect: end-to-end benchmarks are within noise with and without them, while the beams get larger and stack traces lose inlined frames.
This was referenced Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commits, driven by eprof under load (64 callers, pool of 16, arithmetic test server).
Server loop. The in-flight request queue moves from a shared ETS table into the server state as a map; each request paid two ETS copies plus lock traffic for data only its own process touches. Reply messages now carry just the request id instead of the whole cast record (
{shackle_reply, RequestId, Reply}— breaking for anyone matching the raw message instead of usingreceive_response), and telemetry can be disabled with thetelemetryapp env. 156-158k -> 167-168k requests/s (+7%).shackle_socket. A new opt-in protocol built on the socket module. gen_tcp's send path (port_command, port monitor/demonitor, inet_reply round trip, inet_db lookups) was ~32% of server CPU in the profile; socket:send is a single NIF call, and
{otp, select_read}gives active-mode-like delivery with one recv per wake — the profile shows 0.41 recvs per request thanks to reply batching. 168-169k -> 172-174k requests/s (+2.5% over shackle_tcp on the same build). Runtime needs OTP 27.3; the module compiles on the whole CI matrix and dialyzer suppressions cover the pre-28 socket specs. Default protocol is unchanged.Combined: +10% end to end.